Formal Theories of Language Acquisition: Practical and Theoretical Perspectives

Learning Theory is the study of systems that implement functions from ev ident ia l states to theor ies . The theoret ica l framework developed in the theory makes possible the comparison of classes of algorithms which embody d i s t i n c t learning strategies along a var ie ty of dimensions. Such comparisons y ie ld valuable information to those concerned with inference problems in Cognit ive Science and A r t i f i c i a l I n te l l i gence . The present paper employs the framework of Learning Theory to study the design spec i f icat ions of induct ive systems which are of in terest in the domain of language acqu i s i t i on . Section 0: In t roduct ion Learning Theory is tho invest igat ion of systems that implement functions from ev ident ia l states to theor ies . Of central concern is the character izat ion of condit ions under which such funct ions s tab i l i ze to accurate theories of a given environment. Within the theory, the informal notions of "evidence," " theory," " s t a b i l i z a t i o n , " "accuracy," and "environment" are replaced by precise d e f i n i t i o n s . A l ternat ive formulations of these concepts y i e l d a l te rna t ive models w i th in the theory. The vigorous development of Learning Theory began with a celebrated paper by Gold (1967). Angluin & Smith (1982) provide a valuable review of formal resu l t s . Learning Theory is motivated by both s c i e n t i f i c and technological concerns. S c i e n t i f i c a l l y , the theory has proved useful in the analysis of human learn ing , p a r t i c u l a r l y , language acqu is i t ion (see Osherson, Stob & Weinstein, forthcoming, for a review of issues) . Technological ly, the theory helps specify what is learnable 1n p r i n c i p l e , and may thus guide the construct ion of p rac t i ca l systems of induct ive inference. Learning Theory y ie lds po ten t ia l l y valuable insights about problems of inductive inference in the context of Cognitive Science and A r t i f i c i a l I n te l l i gence . The theory provides the framework for systematic comparison of various learning a lgor i thms. Such comparisons are pa r t i cu la r l y useful in determining the re la t i ve strength of classes of algorithms which embody d i s t i n c t abstract learning s t ra teg ies , in assessing thei r resource requirements, and in predic t ing thei r behavior in various environments. W h e n combined with empir ical studies of language acqu i s i t i on , Learning Theory may provide constraints on the character of the learning strategies implemented by ch i l d ren , and re f lec t in turn on the character of the class of languages which m a y be a c q u i r e d . Such studies in C o g n i t i v e Science m a y be of importance to system bui lders in A r t i f i c i a l I n te l l i gence . They suggest that the search for ideal learning s t ra teg ies ' is not well mot ivated. Rather, by focussing on learners who embody d i f fe ren t "s ty les" of learn ing , and by invest igat ing thei r p roper t ies , the theory allows a comparison of the opt imal i ty of d i s t i n c t approaches to learning along a mult i tude of dimensions. In add i t i on , through the analysis of classes of algorithms that embody d i s t i n c t learning s t ra teg ies , t h i s t h e o r e t i c a l framework m a y provide a useful complement to studies of ad hoc systems b u i l t to perform induct ive inference in problem domains of l im i ted scope. The present paper reviews some of our recent work on p rac t i ca l inference and relates it to problems in language acqu i s i t i on . We consider design spec i f ica t ions for Inductive systems relevant to (a) the speed of Inference, (b) the s imp l ic i t y of in fer red theor ies , (c) the l i ke l ihood of i n f e r e n t i a l success, and (d) the res i l ience of such systems in environments subject to informational imperfect ion. At tent ion D. Osherson et a l . 567 is r e s t r i c t e d to l e a r n i n g paradigms in which only " p o s i t i v e i n f o r m a t i o n " i s a v a i l a b l e about the language or d a t a s e t to be i n f e r r e d ; d i r e c t i n f o r m a t i o n about nonmembership is not o f f o r e d to the l e a r n e r ; A n g l u i n & Smith (1982) survey r e s u l t s r e l e v a n t to l e a r n i n g paradigms in which both p o s i t i v e and n e g a t i v e i n f o r m a t i o n is assumed a v a i l a b l e . Our e x p o s i t i o n is o r g a n i z e d as f o l l o w s . The next s e c t i o n p r o v i d e s d e f i n i t i o n s and c o n s t r u a l s at the h e a r t of contemporary l e a r n i n g t h e o r y . S e c t i o n 2 e x h i b i t s theorems p r o p e r to the t o p i c s (a) (d) l i s t e d a b o v e . P r o o f s of these theorems can be f o u n d in O s h e r s o n , Stob & W e i n s t e i n ( 1 9 8 2 , 1 9 8 3 d ) . In S e c t i o n 3 we c o n s i d e r the r e l a t i o n between r e s u l t s r e v i e w e d here and language a c q u i s i t i o n by c h i l d r e n . 1.2 Learning funct ions Let G be a f i x e d , c o m p u t a b l e i s o m o r p h i s m between SEQ and N. A learning function is any f u n c t i o n f r o m N i n t o M; such a f u n c t i o n w i l l be t h o u g h t of as o p e r a t i n g on members of SEQ ( v i a G) , y i e l d i n g i n d i c e s f o r r e c u r s i v e l y enumerable s e t s . L e a r n i n g f u n c t i o n s may be t o t a l or p a r t i a l , r e c u r s i v e or n o n r e c u r s i v e . The ( p a r t i a l ) r e c u r s i v e l e a r n i n g f u n c t i o n s are j u s t Φ, Φ 1 , . . . , Φ The c l a s s of a l l l e a r n i n g f u n c t i o n s is d e n o t e d : F. The c l a s s of a l l r e c u r s i v e l e a r n i n g f u n c t i o n s ( p a r t i a l or t o t a l ) is d e n o t e d : F r e c . 1.3 Convergence, i d e n t i f i c a t i o n Section 1: The Gold modol 1.1 Sequences, languages, texts N is the set of natural numbers. We take the notions f i n i t e sequence ( i n N) and i n f i n i t e sequence ( i n N) to be basic. The set of a l l f i n i t e sequences is denoted: SEQ, For n £ N, and i n f i n i t e sequence, t: r n g ( t ) is the set of numbers appearing in t; t n is the nth member of t ; and t n is the f i n i t e sequence of length n in t. Let P0. P1 pi,... be a f i x e d l i s t o f a l l p a r t i a l recursive functions of one variable, and assume the 1 i s t to be acceptable in the sense of Rogers (1967, Ch. 2). For i £ N, l e t W1 domain pi, the recursively enumerable subset of N with Index i, Languages are I d e n t i f i e d with nonempty members of {W1 € N). The c o l l e c t i o n of a l l languages is denoted: . For L £ , 1 C N, 1f L = Wi then 1 1s said to be f o r L. S e c t i o n 2 : P r a c t i c a l l e a r n i n g 2 . 1 E f f i c i e n t i n f e r e n c e A t e x t for L £ &6, Is any I n f i n i t e sequence such that r n g ( t ) = L, The class of a l l texts f o r L is denoted: TL. Given a c o l l e c t i o n , L, "T, denotes uL € L of languages.